Papers by Sarah Masud Preum
REGen: A Reliable Evaluation Framework for Generative Event Argument Extraction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work evaluates event argument extraction with exact match (EM), where predicted arguments must align exactly with annotated spans. |
| Approach: | They propose a Reliable Evaluation framework for Generative event argument extraction that combines exact, relaxed, and LLM-based matching to better align with human judgment. |
| Outcome: | Experiments on six datasets show that REGen achieves an average performance gain of +23.93 F1 over EM, reflecting capabilities overlooked by prior evaluation. |
How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Drafting (2026.acl-long)
Copied to clipboard
Parker Seegmiller, Joseph Gatto, Sarah E. Greer, Ganza Belise Isingizwe, Rohan Ray, Timothy E. Burdick, Sarah Masud Preum
| Challenge: | Large language models (LLMs) have been shown to be effective in drafting patient portal responses, yet their integration into clinical workflows raises various concerns. |
| Approach: | They propose a taxonomy of thematic elements in clinician responses and a framework for assessing clinician editing load of LLM-drafted responses at both content and theme levels. |
| Outcome: | The proposed framework assesses the editing load of LLM-drafted responses at both content and theme levels. |
Explicit, Implicit, and Scattered: Revisiting Event Extraction to Capture Complex Arguments (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on event-specific argument extraction is limited to contiguous spans of text . Existing approaches to event-centric information extraction are limited to explicit arguments . |
| Approach: | They propose two key argument types that cannot be modeled by existing EE frameworks . implicit and scattered arguments are crucial to elicit full breadth of information required for proper event modeling. |
| Outcome: | The proposed dataset includes 7,464 argument annotations from online health discourse. |
Deciphering Hate: Identifying Hateful Memes and Their Targets (2024.acl-long)
Copied to clipboard
| Challenge: | a growing body of research has focused on the negative aspects of memes in high-resource languages like Bengali . a new dataset for Bengali hateful memes is designed to detect their targeted entities . |
| Approach: | They propose a multimodal dataset that analyzes the modality of memes and compares them with other datasets. |
| Outcome: | The proposed dataset outperforms state-of-the-art datasets on Bengali hateful memes . the proposed dataset is generalizable on other low-resource hateful memes datasets compared with baselines based on the proposed model . |
Document-Level Event-Argument Data Augmentation for Challenging Role Types (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Event Argument Extraction (EAE) are not well-suited to a variety of real-world situations, including long documents and challenging role types. |
| Approach: | They propose two novel methods for generating document-level EAE samples using zero in-domain training data and validate their generalizability. |
| Outcome: | The proposed methods show significant performance increases in low-resource settings. |
Follow-up Question Generation For Enhanced Patient-Provider Conversations (2025.acl-long)
Copied to clipboard
Joseph Gatto, Parker Seegmiller, Timothy E. Burdick, Inas S. Khayal, Sarah DeLozier, Sarah Masud Preum
| Challenge: | Follow-up question generation is an essential feature of dialogue systems as it can reduce conversational ambiguity and enhance modeling complex interactions. |
| Approach: | They propose a framework that generates personalized follow-up questions based on patient utterances and prior EHR data. |
| Outcome: | The framework reduces follow-up communications by 34% and improves performance by 17% and 5% on real and synthetic data. |
A Corpus of Drug Usage Guidelines Annotated with Type of Advice (L18-1)
Copied to clipboard
| Challenge: | Current research indicates patients are often unaware of such critical information / advice related to their prescription drugs due to lack of communication with their doctors and/or pharmacists. |
| Approach: | They propose an annotation scheme for annotating safety critical advice from drug usage guidelines and an annotated dataset containing drug usage guideline data. |
| Outcome: | The proposed dataset will accelerate further release of annotated drug usage guideline datasets and research on automatically filtering safety critical information from these documents. |
Measuring Distribution Shift in User Prompts and Its Effects on LLM Performance (2026.acl-long)
Copied to clipboard
| Challenge: | a large-scale evaluation of deployed LLMs under natural prompt distribution shift is needed . natural prompt behavior shifts can cause performance degradation in dynamic, real-world settings . |
| Approach: | They propose a data-centric framework for measuring natural prompt distribution shift . they train models on 4.68M training prompts and evaluate on 57.6k prompts . |
| Outcome: | The proposed framework evaluates natural prompt distribution shift in LLMs over time and between user groups. |
Align before Attend: Aligning Visual and Textual Features for Multimodal Hateful Content Detection (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to multimodal hateful content detection focus on detecting hate speech from text-based content, but they fail to address modality-specific features. |
| Approach: | They propose a context-aware attention framework for multimodal hateful content detection that integrates an attention layer to meaningfully align the visual and textual features. |
| Outcome: | The proposed framework achieves F1-scores of 69.7% and 70.3% on two hateful meme datasets and shows 2.5% and 3.2% performance improvement over the state-of-the-art systems. |